Grade: Gibbs Reaction and Diiusion Equation { a Framework for Pattern Synthesis, Denoising, Image Enhancement, and Clutter Removal

This paper proposes a new class of nonlinear PDEs called the Gibbs Reaction And Di usion Equation GRADE for a variety of applications in computer vision image processing and graphics In two previous papers the authors have been studying a minimax entropy theory based on which a new class of Gibbs distributions are learned in a fully non parametric form from a set of observed images such as textures and natural images These Gibbs distributions have potentials of the form U I S PK P x y F I x y with S fF F F K g being a set of lters and f K g the potential functions We nd that the learned s can be divided into two categories and subsequently the partial di erential equations given by gradient descent on U I S are essentially reaction di usion equations where the energy terms in one category produce anisotropic di usion while the energy terms in the second category produce reaction associated with pattern formation enhancing preferred image features This paper aims to provide a statistical framework of designing reaction di usion equations based on training images in a given application instead of simulating physical and chemical processes as previous methods did We demonstrate experiments where the GRADE is used for texture pattern formation denoising image enhancement and clutter removal Introduction Nonlinear partial di erential equations PDEs have long been adopted in modeling a vast variety of phenomena in chemistry physics and biology In the past decade some of these PDEs have inspired interesting methods for solving problems in computer vision image processing and graphics among which the following two examples are remarkable I The Turing reaction di usion equations for modeling the chemical mechanism for mammalian coat patterns Turing such as leopard blobs and zebra stripes Recently these PDEs are utilized to render realistic textures in graphics Turk Witkin and Kass and they are also adopted in image processing such as enhancing ngerprints and halftoning Price Sherstinsky and Picard II The Perona Malik anisotropic di usion equations Perona and Malik These equations are used for generating scale space and removing noise Nitsberg and Shiota Despite the success of these PDEs many questions are not understood in the literature Firstly the strengths of these PDEs lies on their nonlinearity which also makes the equations unstable or unbounded especially when reaction terms are involved Secondly these PDEs are derived from chemical or physical models and there is little justi cation for applying them to images which seems irrelevant to the original chemical or physical processes Thus the selection of the PDEs and their coe cients are subjective Thirdly there lacks a generic theory to guide the design of PDEs for general purposed applications in computer vision and image processing This paper proposes a statistical framework for designing a new class of PDE called the Gibbs Reaction And Di usion Equation GRADE inspired by our previous work on modeling texture and natural images In two previous papers the authors have been studying a minimax entropy theory based on which a Gibbs distribution p I is learned from a set of observed images p I has potential of the form U I S PK P x y F I x y where S fF F F K g is a set of lters which best characterize the set of observed images and f K g are the potential functions which are learned in a non parametric form such that p I reproduces the observed statistics captured by S The learned Gibbs distributions were veri ed by stochastic sampling in the spirit of analysis synthesis i e if p I models the observed images correctly the images sampled from p I should have similar appearances to the observed ones We nd that the learned s in p I can be divided into two classes and sub sequently the partial di erential equation given by gradient descent on U I S is essentially reaction di usion equation which we named Gibbs Reaction And Di usion Equation GRADE The GRADE consists of two components the energy terms from one class produce anisotropic di usion while the energy terms in the second class produce reaction associated with pattern formation enhancing preferred image features Unlike the PDEs derived from a chemical or physical process GRADE is well learned from a given application and their behaviors are speci ed by the Gibbs probability distributions We demonstrate experiments where the GRADE is used for texture pattern formation denoising image enhancement and clutter removal This paper is organized as follows Section brie y reviews previous work on non linear PDEs Section derives the Gibbs reaction and di usion equation and analyzes its properties Section presents experiments on pattern synthesis denoising image enhancement and clutter removal Then section concludes with a discussion Related work This section reviews some of the nonlinear PDEs previously used in computer vision Reaction di usion model for texture pattern formation A set of nonlinear PDEs was rst studied in Turing for modeling the formation of animal coat patterns by the di usion and reaction of chemicals which Turing called the morphogens These equations were further explored by Murray in theoretical biology Murray For example let A x y t and B x y t be the concentrations of two morphogens at location x y and time t the typical reaction di usion equations are A t Da A Ra A B B t Db B Rb A B whereDa Db are constants x y is the Laplacian operator andRa A B Rb A B are nonlinear functions for the reaction of chemicals e g Ra A B A B A and Rb A B A B The morphogen theory is by no means accepted by theoretical biologists because no such morphogen has been identi ed however they provide a way for synthesizing some texture patterns Turk Witkin and Kass In the texture synthesis experiments chemical concentrations are replaced by various colors and the equations run for a nite steps with free boundary condition starting with some initial patterns In some cases both the initial patterns and the running processes have to be controlled manually in order to generate realistic textures Two canonical textures synthesized by the Turing reaction di usion equation are the leopard blobs and zebra stripes Recently these reaction di usion equations are utilized in image processing such as enhancement of ngerprint images and image halftoning Price Sherstinsky and Picard Another set of PDEs for pattern formation was studied by Swindale in modeling the development of the binocular dominance stripes in the visual cortex of cats and monkey Swindale and the simulated patterns are very similar to the zebra stripes Again such theory is not veri ed in experiments of neurophysiology In the reaction di usion equations above reaction terms are responsible for pattern formation however they also make the equations unstable or unbounded and human interference is needed Even for a small system the existence and stability problems for these PDEs are intractable Grindrod In fact we believe that running any nonlinear PDEs for a nite steps will rendering some patterns and blobs and stripe appear to be the most canonic ones but it is unknown how to design a set of PDEs for a given texture pattern Anisotropic di usion Perona and Malik introduced a family of anisotropic di usion equations for generating image scale space I x y t Perona and Malik Their equation simulate the heat di usion process It div c x y t rI I x y I where rI Ix Iy is the intensity gradient and div is the divergence operator div V rxP ryQ for any vector V P Q In practice the heat conductivity c x y t is de ned as function of location gradients for example Perona and Malik chose It rx Ix b Ix ry Iy b Iy where b is a scaling constant It is easy to see that equation minimizes the following energy function by gradient descent U I Z Z rxI ryI dxdy where const log b and const b are plotted in gure Sim ilar form of the energy functions are widely used as prior distributions in image restoration and segmentation Geman and Geman Geman and McClure Blake and Zis serman Mumford and Shah Geiger and Girosi −30 −20 −10 0 10 20 30 −1 0 1 2 3 4 5 6 x −30 −20 −10 0 10 20 30 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1